Journal of Vision

The critical flicker fusion (CFF) threshold is a psychophysical measure used to quantify the temporal resolution of the visual system and is known to vary across individuals. However, it is unclear if this measure is stimulus-specific, or if it may represent a more fundamental processing rate for visual perception in general. Here, we assess if individual variation in CFF is predictive of two features of visual processing that are dependent on temporal perception: the attentional blink and global motion sensitivity. In a non-clinical sample of 84 individuals, flicker fusion thresholds were predictive of the magnitude of the attentional blink. In contrast, we found no link between flicker fusion and global motion sensitivity in a sample of 79 individuals. Our results suggest that CFF reflects a visual processing rate that impacts other, more complex perceptual tasks.

Mounting evidence suggests that recurrent processes in the visual system play a critical role during challenging recognition tasks. Backward masking techniques have traditionally been used as a non-invasive method for studying recurrent processes: A mask follows the target image, presumably disrupting ongoing processes. However, these techniques have the limitation that they do not allow the identification of the stage of the visual system at which critical recurrent processes are taking place. Here, leveraging advances in texture synthesis via deep networks, and the approximate correspondence between stages of the visual system and layers of deep networks, we develop a novel psychophysics paradigm where masks with textures targeting different stages of the visual system follow the presentation of challenging images. In a series of experiments, we present objects to human subjects either for a short duration or in unusual poses, followed by a textured mask either designed to only target the early visual system, or the entire visual system. We find that both texture types equally affect recognition abilities, suggesting that recurrent processes in or towards early stages of the visual system are already recruited for these recognition tasks.

Blinks are a ubiquitous yet largely unnoticed aspect of human vision, despite causing frequent interruptions of visual input that can amount to up to 10% of waking time. By leveraging a large dataset of unconstrained gaze behavior during two natural tasks, we found a novel behavioral strategy to limit the impact of blinks: blinks were strategically coupled with head movements, which minimizes information loss due to unreliable visual input during head movements. Specifically, blink probability increased with higher head velocities and showed strong temporal modulation relative to head movement onset. Blink probability was reduced before head movement initiation and then peaked during the head movement. The strength of this coupling was tailored to the individual needs of participants, with participants with higher baseline blink showing a stronger synchronization. This indicates that blinks are a part of an individually coordinated strategy when orchestrating eye and head movements during unconstrained natural behavior.

The ability to mentally rotate objects is a fundamental feature of human cognition, and humans can use this ability to make choices about objects based on their geometry. However, remarkably little is known about how such choices are reached, and what sort of visual information might facilitate them. We devised an experiment where participants had to mentally simulate an object's rotation to choose which of two objects was better for a subsequent task based on its shape alone. We also tracked their gaze while they made their choice, to see which visual information they were using to facilitate this mental simulation. We found that participants were consistently able to choose the most suitable object for the task, and, remarkably, the visual information they sampled was directly linked to their choices. Put simply, participants made better choices when they looked at more informative regions of the objects, and participants who sampled regions that were better for facilitating mental simulation made better choices overall. These findings reveal a direct link between fixations, simulation, and decision-making, suggesting that to perform any fine-grained mental simulation people need to direct their gaze at specific, informative points of an object to simulate its two-dimensional proximal image displacement.

The dynamic range of contrast-encoding in the early visual system has been investigated at both single-neuron and population levels in animals using oriented stimuli such as gratings and Gabor patches. However, contrast-encoding of unoriented stimuli such as Gaussians has been less explored, even though such stimuli can evoke a large population response. In studies that employ Gaussians, contrast response functions (CRFs) of neurons and neural populations in primary visual cortex are typically characterized across a limited range of Weber contrasts up to 100%, which may not adequately reflect the statistics of contrast in natural scenes. Indeed, our analysis shows that locations with contrasts far exceeding 100% Weber contrast are ubiquitous in natural scenes. Thus, our current understanding of contrast encoding remains incomplete in the context of natural environments. Here, we measured spiking activities of individual neurons using electrophysiology and population responses using calcium imaging in V1 of fixating macaques while Gaussian stimuli were presented over a wide range of Weber contrasts, up to 900%. Both the average spiking response and the average population response continued to increase robustly above 100% Weber contrast. Only a small minority of neurons saturate below 100% Weber contrast. These results demonstrate that the dynamic range of contrast-encoding in V1 is broader than previously assumed and aligns more closely with the statistics of contrast in natural scenes. Significance StatementThe dynamic range of contrast-encoding in V1 has typically been characterized using only a limited range of contrasts, often excluding the high contrasts commonly found in natural scenes. We measured contrast response functions of individual neurons and neural populations in V1 across a much wider range of contrasts. We found that the responses of the majority of neurons, as well as the overall population, increased robustly up to extremely high contrasts. These findings suggest that the dynamic range of contrast encoding in V1 extends well beyond the commonly tested range.

Representations of the same physical stimulus vary between individuals. Characterizing individual differences has practical implications, but is challenging because these representations are not directly observable. Given a model of how representations vary within a population, we propose a Bayesian adaptive procedure for estimating an individual observer's representation from a series of targeted perceptual discrimination judgments. A key component of our approach is using Fisher information to identify stimulus distortions that efficiently differentiate observers in the population. As a proof of concept, we focus on individual differences in color perception and simulate observers with cone fundamentals drawn from an individual colorimetric observer model. We demonstrate that our approach can recover key aspects of a sampled observer's cone fundamentals using simulated three-alternative forced-choice oddity judgments with approximately 500 trials, corresponding to an experimental duration of approximately one hour. Our Bayesian adaptive framework provides a promising and generalizable approach to efficiently link behavioral measurements to individual differences in sensory representations.

During sideways movement of an observer, optic flow parsing - in which an objects speed in the world is extracted from all the other visual movement present in the scene, self-generated and otherwise - has been shown to be incomplete, leading to biases in speed perception, particularly when object and observer are moving in opposite directions. Here, we assess how judgements about the speed of objects moving in depth (judged relative to the world) towards or away from an observer (6 m/s) are affected by simultaneous movement of the observer either in the same or opposite direction as the object. In a virtual reality display, participants (n = 25) viewed a sphere simulated as moving in a corridor either while they were stationary or during visually simulated self-motion in the same or opposite direction as the object. They judged the spheres movement relative to the world by comparing its motion to a probe sphere that travelled laterally across the corridor in front of them. In a second experiment (n = 28) participants performed the same task but during faster self-motion (10 m/s). The second cohort also judged the direction in which the object was perceived to move during the same combinations of self and object speeds. Object speed was overestimated when the object travelled in the direction opposite to the observer compared to how objects motion was judged when the observer was stationary. However, object speed was also overestimated during self-motion in the same direction as the object where participants were also much more likely to misjudge the direction of motion of the object. Precision of judgements was lower when self-motion was simulated than it was for stationary observers. A simple arithmetic model of flow parsing fails to capture these results satisfactorily, suggesting that different mechanisms may be at play when the observer travels in the same direction as a moving object and is vulnerable to misperceiving its direction of travel.

The human visual system segments images using both high-level recognition mechanisms and low-level mechanisms that are largely independent of specific prior experience. The low-level mechanisms are essential for initiating recognition processes, and for learning to recognize new materials, objects, and contexts. Here we describe a hierarchical Bayesian observer (HBO) model of texture segmentation that is biologically plausible, takes into account the statistics of natural scenes, and does not depend on prior experience. The HBO model consists of five steps: local similarity grouping with local normalization, mutual similarity grouping (local grouping is strengthened if the neighboring regions are similar to the same set of other regions), transitive grouping (good continuation), confidence grouping (neighboring regions far from the same-different decision boundary guide grouping of regions near the decision boundary), and region grouping (similarity grouping of the regions from the initial segmentation). We find that a local similarity grouping process, trained to maximize accuracy, predicts human texture discrimination accuracy. We then find that the four additional steps accurately segment images with randomly shaped regions containing arbitrary natural textures. The success of the model depends on all the steps, but especially on local-similarity and transitive grouping. We also find that the transitive grouping allows correct segmentation of non-stationary texture regions (e.g., textures slanted in depth). Further, we find that when illumination varies across the image, local normalization enables both correct texture segmentation and estimation of illumination change. Finally, we find that unlike our model large state-of-the-art deep networks often fail on these stimuli.

Human perception is shaped by both sensory input and prior knowledge or expectations. But how does prior contextual information influence rapid visual processing? Here, we combined eye tracking with feature-based encoding models across two experiments to predict detection latencies in a core visual task: rapid face detection in natural scenes (N = 38 per experiment). In the first experiment, we manipulated the presence of faceless scene previews. In the second experiment, we additionally restricted peripheral visual input using a moving-window paradigm, thereby increasing reliance on prior information. Across both experiments, prior context facilitated face detection, particularly for challenging images. This facilitation was already evident in the very first eye movement, suggesting that previews shape perceptual strategies from the outset. To quantify what information guided behavior, we modeled detection latencies using a set of image-based predictors capturing (i) sensory information and (ii) a scene-derived spatial prior: the expected face location. Both predictor classes explained latency variation across images. Among sensory predictors, the difference in deep neural network responses induced by the presence of the face provided the strongest out-of-sample prediction of detection latency. Critically, when scene previews were available, the contribution of the spatial prior increased, while reliance on sensory-driven features was generally reduced. Together, these findings indicate that prior scene context shifts the balance of information used for rapid face detection from sensory-driven to expectation-based spatial guidance.

Current debates regarding the relative contribution of saliency versus semantics to gaze control often rely on comparing the predictive power of saliency and meaning maps. We argue that such indirect, global approaches are fundamentally limited because fixations arise from heterogeneous, local causes that are conflated in whole-scene comparisons. To substantiate this claim, we used a direct method where participants explicitly identified the reasons for fixation at specific clusters of high fixation density, distinguishing between low-level saliency and various semantic categories, as well as the most important one. The obtained judgments revealed that multiple factors contribute simultaneously to gaze control. Although their influence varied across fixation clusters, semantics generally dominated saliency. Notably, abstract semantic categories, particularly "unknown/unusual," proved important, highlighting the role of prior knowledge and novelty besides personal relevance in guiding attention. To interpret these findings in the context of existing models, we propose a framework distinguishing between processes highlighting interesting locations in the image from a sampling strategy translating this information into scanpaths. Within this framework, classic saliency and meaning maps are viewed as restricted inputs to the strategy, whereas deep learning-based models (e.g., DeepGaze IIE) are more general and may also implicitly encode aspects of the strategy itself. Consistent with this, we found that the predictive performance of DeepGaze IIE varied less significantly with the specific reasons for fixation than that of classic saliency and meaning map approaches.

Saccades made during memory maintenance prioritize memory for the saccade target, but it is unclear if this benefit is specific to a location or extends across memorized objects. In three experiments, we examined whether saccadic selection spreads to other locations within the same object. In Experiment 1, we asked observers to remember three oriented Gabors presented either within contour-defined objects or without object structure. A subsequent movement cue prompted observers to move their eyes to the indicated location. We then probed memory for stimuli at locations equidistant from the saccade target, in either the same or a different object. Memory was best for stimuli at locations congruent with the saccade target, and consistently weaker for other stimuli presented in the same or a different object than the saccade target. In Experiment 2, we created more complex objects by adding more object features to the stimulus. Again, memory performance was best for stimuli congruent with the saccade target location, whereas memory in incongruent trials was worse and similar for stimuli in the same and different object as the saccade target. In Experiment 3, we tested if saccadic selection is present and propagates within the object in a change detection task. Again, memory performance (i.e., change detection) was best at the saccade target location. However, this memory benefit also spread to other locations within the same object. Our results imply that saccadic selection in visual working memory is primarily space-based but can also spread towards locations within the object where a saccade was directed.

Human object recognition is robust to challenging conditions, such as when ones view of an object is fragmented due to an occluding foreground object. In comparison, deep neural networks (DNNs) are typically more susceptible to occlusion, suggesting that human vision relies on distinct mechanisms. Here, we investigated the role of visual diet in the emergence of these mechanisms by asking whether human-like robustness might arise in DNNs when trained with image datasets that better reflect the properties of occlusion in natural vision. We trained convolutional and transformer DNNs to classify clear images only, images augmented with artificial occluders (i.e., geometric shapes) or natural occluders (objects segmented from photographs). We then evaluated DNN occlusion robustness and compared their performance profiles with 30 human participants. We found that DNNs trained with artificial occluders remained vulnerable to natural occlusion and exhibited less human-like performance than those trained with natural occlusion. Our findings suggest that human robustness to visual occlusion arises from learning to disentangle natural objects from each other rather than simply learning to recognize objects from partial views. They also imply that commonly used forms of artificial occlusion are unsuitable for the evaluation or promotion of robustness to real-world occlusion in DNNs.

During active vision, the brain must coordinate where to move the eyes with predictions about upcoming sensory input. Before each saccade, perception is enhanced at the upcoming fixation location, but whether this enhancement depends on expectations about target features remains unknown. Here, participants prepared a saccade to a cued location while reporting the presence and orientation of a brief visual target that appeared either at the saccade goal or at the opposite location. Feature expectation was manipulated across blocks by varying the probability of the two target orientations. Perceptual sensitivity (d') increased when targets were presented at the saccade goal, consistent with presaccadic enhancement, and was also higher for less expected features. However, these effects were independent: feature probability did not alter the magnitude of presaccadic enhancement. Moreover, presaccadic enhancement increased near saccade onset, whereas the advantage for less expected features weakened as movement onset approached. Saccade latency revealed a contrasting pattern. Visual targets presented at the saccade goal delayed movement initiation. This delay depended on feature probability, with longer latencies for unexpected than for expected features only when saccades were directed towards the target. This location-specific effect persisted after accounting for perceptual report, and the latency cost for unexpected features was reproduced in a follow-up experiment. Together, these findings show that feature probability enhanced sensitivity to unexpected information independently of presaccadic enhancement, while selectively delaying saccade initiation towards targets with unexpected features. This dissociation suggests that feature expectation modulates perception and action through functionally distinct forms of visual processing.

Fixational eye movements (FEMs) comprise the involuntary small scale eye motion conducted during fixation on a stationary stimulus. As a consequence, the visual information can be spread across multiple photoreceptors reducing the local signal-to-noise ratio. Yet, the signals transmitted by individual photoreceptors adapt to constant stimulation so that an entirely still scene would eventually fade from view. Because FEMs convert a stationary stimulus in the world to a temporally varying one on the retina, they can act to prevent this stimulus fading. Thus, FEMs can be understood as a sampling protocol than needs to be adjusted to the underlying processing circuitry. We analyse the impact of FEMs on the rate of information acquisition at the level of the retina for two common tasks of the human eye that typically go hand in hand: detection and localization. Here, we build a simple analytical model of visual perception, i.e. we subject a continuous receptor array to a stimulus moving across the retina as a consequence of FEMs with receptor excitations depending on past stimulation through a linear response function. Using Bayesian inference we quantify both the probability of detection and the accuracy of localization as a function of parameters controlling eye movements and stimulus. We find that localization of a stimulus is equivalent to the detection of the stimulus gradient. This allows us to discern optimal properties of eye movements for the respective tasks and provides a link between two typical psychophysical observables: detection thresholds and Vernier acuity. Our analysis suggests that typical human FEMs tend to facilitate localization at the expense of detection. Simply put, if you can see a stimulus you also know where it is. Finally, we propose a variety of experimental protocols to investigate the interplay between FEMs, detection, and localization with the potential of inferring intrinsic properties of an individuals visual system.

Whether and how various visual sensory areas contribute to the perceived location of visual stimuli remains unknown. To test the role of neurons in extrastriate area V4 in generating alterations in spatial perception during saccadic eye movements (saccades), we examined perisaccadic mislocalization--the perceptual phenomenon in which visual stimuli appearing around the time of a saccade are perceived at a different position than their actual location. We designed and implemented a combined behavioral and electrophysiological framework in non-human primates to directly relate trial-by-trial spatial perception reports during saccades to neuronal firing rates in V4 populations. We measured monkeys perception of stimulus location behaviorally and found perisaccadic mislocalization opposite to the saccade direction. We also quantified population responses by computing the center of mass of firing rate activity across probe locations for V4 neurons with receptive fields close to the saccade target. While perisaccadic neuronal responses showed shifts toward the saccade target, these shifts did not systematically vary with the magnitude of perceptual mislocalization across trials. In conclusion, receptive field shifts based on the perisaccadic firing rate of V4 neurons are not sufficient to account for the magnitude of perceptual mislocalization in each trial, suggesting that more complex neural representation of perisaccadic visual information may be critical for linking extrastriate neural activity to saccade-induced perception.

Visual signals from the two eyes merge gradually as they pass through the primary visual cortex (V1). Here we use a computational model of Macaque V1 to study the first stage of this integration along the magnocellular pathway, in layer 4C, aiming to infer neuroanatomical origins of binocular response. It is known that neurons in layer 4C are predominantly monocular, though some do exhibit varying degrees of binocularity. We find (1) the emergence of narrow binocular strips along borders of ocular dominance columns (ODC), a finding that aligns with experiments; (2) most consistent with data is when 10 - 30% of interactions near ODC boundaries are cross-columnar; and (3) feedback from layer 6 is largely monocular. These results were obtained through systematic hypothesis testing using a multiscale model that is orders of magnitude faster than its biologically-detailed predecessors. We propose that multiscale modeling can be an effective tool for bridging anatomy and function.

Recent work has brought to light a number of stimulus families whose perception is shaped by strong idiosyncratic biases. These biases differ significantly from one observer to the next, yet remain quite stable within observers when measured over multiple points in time, sometimes over months or even years. Nevertheless, we have previously shown that at least some of these biases undergo small but systematic changes over time. Although these temporal changes are also idiosyncratic, they generally act as a kind of memory that accumulates small random steps. Other research has shown that when stimuli are shown at different points in the visual field, biases can vary idiosyncratically across the spatial field as well. Here we ask whether variations in biases across space follow any regular pattern, and whether spatial and temporal variations are independent of one another. Measuring biases for surface orientation in structure-from-motion stimuli, sampled at numerous points in space and time, we find that variations both in time and space are positively autocorrelated: the closer two points are to each other in space or in time, the more similar the biases at those two points. We also found that spatial and temporal variations of bias are correlated, both between and within participants. Bias variations over space, time, and space-time are therefore not random but follow dynamics that may provide clues about the underlying mechanisms.

Numerosity adaptation (the underestimation of number after exposure to a numerous adaptor) is reduced when adaptor and test differ in color, suggesting that the numerosity system parses items into color-defined categories. Here we ask whether this chromatic selectivity is organized into multiple narrowly tuned chromatic channels, and whether its expression depends on individual chromatic sensitivity. Twenty observers (aged 22-61) completed two psychophysical tasks. First, chromatic discrimination was measured for five hues spaced in 5{degrees} CIE L*a*b* steps ({Delta}H = 0{degrees}, 5{degrees}, 10{degrees}, 15{degrees}, 20{degrees}) from a red reference (LCh: 54, 118, 38), yielding an individual just-noticeable difference (JND). Second, numerosity adaptation was measured across the same five chromatic distances between a 48-dot adaptor and the test. Observers with superior discrimination (JND < 2.5{degrees}) showed robust chromatic tuning, adaptation declining as the test moved away from the adaptor hue, whereas poorer discriminators showed none. Using an interindividual-covariance / factor-analytic approach, we found that adaptation strengths at neighboring chromatic distances were highly correlated and fell off with chromatic separation. Principal component analysis extracted two factors, one loading on the larger chromatic distances and one on the smaller; under oblique (promax) rotation the two factors were substantially correlated (r = .66), implying at least two dissociable but overlapping chromatically tuned mechanisms. These results suggest that numerosity adaptation is mediated by multiple, comparatively narrow chromatic channels, resembling the higher-order color mechanisms inferred from color scaling, SSVEP, and fMRI, rather than the two early cardinal axes (L-M, S-(L+M)).

Continuous Psychophysics, which couples a continuous stimulus with a continuous response, is a promising tool to break out of the confines of traditional designs based on discrete trials. In this pre-registered study, we explore to what extent this paradigm is useful in the study of multisensory integration. We expand on Tonelli et al.s (2025) seminal study by additionally examining the role of eye-movements, using a Kalman filter to estimate the sensory noise underlying behavioral tracking parameters and employing a virtual reality set-up. We immersed two cohorts of participants (n = 30 each) in a virtual meadow environment and asked them to continuously track a drone (Experiment 1) or a swarm of flies (Experiment 2) with a controller, while simultaneously recording their eye movements. We manipulated the reliability of visual cues using four levels of fog (from a completely clear view to impenetrable fog where no visual cues to the targets position were available) as well as the presence of sound cues emitted from the object (sound present/absent). The maximum correlation between stimulus and response was higher when sound was present in some conditions, particularly when visual uncertainty was high, while the tracking delay remained unaffected across all fog levels. Using a Kalman filter to estimate the underlying sensory noise, we found strong evidence that sensory noise was lower when sound was present than when sound was absent both for manual and for ocular tracking, particularly for those conditions with higher visual uncertainty. In exploratory analyses, we further show strong correlations between manual and ocular tracking in all measures (maximum correlation, tracking delay, sensory precision). However, when isolating the multisensory advantage, these correlations all but disappeared for maximum correlation and tracking delay, while remaining substantial for sensory precision. Similarly, behavioral tracking correlated generally strongly with underlying sensory noise, but much less so when it came to the advantage conferred by added sound cues. Our results show that continuous psychophysics is well-suited for the study of multisensory integration, particularly when a Kalman filter analysis is used to estimate sensory uncertainty from behavioral data.

Because of the uncertainties present in the any sensory inflows, we experienced perceptual biases that often contaminate sensorimotor transformations. How these biases can be prevented through perceptual learning or adaptation is an open question in inference theories of perception and motor control. We revisited this debate using a classic example of low-level perceptual bias, the aperture problem in motion perception and its consequences for smooth pursuit. In a visuomotor tracking paradigm, participants pursued left-, right-tilted, or upright Gabor patches moving horizontally. As expected, initial pursuit direction was biased toward the oblique directions. This initial pursuit error remained unchanged when participants repeatedly tracked a single tilted Gabor during training sessions conducted over several days. By contrast, post-training error correction became faster for the trained Gabor orientation, but not for its untrained, mirror-symmetric counterpart. This change in dynamics was explained by the emergence of a delayed compensatory pursuit response, best revealed after training with an upright Gabor, a stimulus that would normally elicit unbiased tracking. This corrective eye movement was selective for both shape and motion features. It lagged by [~]40 msec after pursuit onset suggesting that it was triggered by the biased motor command itself rather than by visual reafference. These results support a dissociation between low-level sensory computations, which cannot be modified over short timescales, and internal models of sensorimotor transformation, which can be rapidly updated to compensate for irreducible but predictable sensory-driven perceptual biases.